None exec_jwst_source_catalog_miri_test

JWST Pipeline Validation Notebook: calwebb_image3: source_catalog with MIRI

**Instruments Affected**: MIRI, NIRCam

Tested on MIRI Simulated data

Table of Contents


[Introduction](#intro_ID)
[Imports](#imports_ID)
[Run Pipeline](#pipeline_ID)
[Check Results](#output_ID)
[About This Notebook](#about_ID)

Introduction

This notebook processes a set of images through calwebb_image2 and calwebb_image3 and examines the output table of the source_catalog step.

This test uses simulated MIRI F770W (options for F560W or F770W) data of a crowded star + galaxy field.

The pipeline documentation can be found here: https://jwst-pipeline.readthedocs.io/en/latest/jwst/source_catalog/main.html

The pipeline code is available on GitHub: https://github.com/spacetelescope/jwst

JWST Calibration working group algorithm

The algorithm and discussion for this step can be found at the following page:

https://outerspace.stsci.edu/display/JWSTCC/Vanilla+Point+Source+Catalog

Test description

The steps of this test are as follow:

1) Set up data path and directory and image files name.

2) Run output of calwebb_detector1 through calwebb_image2.

3) Run output of calwebb_image2 through calwebb_image3.

4) Read in output table of source_catalog step and print ecsv table

5) Display image and overplot detector sources from ecsv table.

6) Look at plots of total flux in Jy and AB mag.

7) Look for matches between expected source positions (RA and Dec) from simulated catalog to output from source_catalog.

8) Compare magnitudes and magnitude differences between input simulated catalog and output found sources.

Data description

The data used in this test consist of a set of four simulated images in the F770W filter of MIRI, at four different dither positions. The data have 4827 stars of varying flux levels and just over 200 galaxies in the image. The MIRI Image simulator (MIRISim) was used to create the simulations.

Create a temporary directory for all of the test data

Set up import statements

The following packages are needed to enable this notebook to run:

Top of Page

Put in some helper scripts

Read in data from artifactory

Run Pipelines

Read in the data and run the JWST calibration pipelines.

Run output of calwebb_detector1 through calwebb_image2

Run output of calwebb_image2 through calwebb_image3

Create an association file to combine the input calibrated files

Run Calwebb_Image3 pipeline

For MIRI, the FWHM values are dependent on filter and should be set using the table below:

Filter FWHM
F560W 1.636
F770W 2.187
F1000W 2.888
F1130W 3.318
F1280W 3.713
F1500W 4.354
F1800W 5.224
F2100W 5.989
F2550W 7.312
F2550WR 7.312

For the fit geometry keyword, the following options are available: fitgeometry: A str value indicating the type of affine transformation to be considered when fitting catalogs. Allowed values:

Results

Read in the output of the pipeline and check your results.

Read in output table of source_catalog step and print ecsv table

Display image and overplot detector sources from ecsv table

In the image above, check that the stars found by source catalog (red) overlap with stars in the image

Look at the fluxes of the sources found

Look at AB mag of sources found

Look at the errors on the AB magnitudes

Manually find matches

Since this is a simulated data set, we can compare the output catalog information from the pipeline with the input catalog information used to create the simulation. Grab the input catalog x,y values and the output catalog x and y values.

Use MIRI software to put x, y coordinates from input catalog into RA, Dec coordinates

Compare RA, Dec coordinates between simulator catalog and output catalog to find matches

Use photutils to find catalog matches and compare which sources were found and matched.

Photutils includes a package to match sources between catalogs by providing a max separation value. Set that value and compare the two catalogs.

Now, catalog_in_matches and catalog_out_matches are the matched sources in catalog_in and catalog_out, respectively, which are separated less than our max_sep value.

Plot RA and Dec positions of both catalogs

The simulated positions are in red and the sources found with source catalog are marked in blue. Matched sources from both are marked in green.

Convert matched RA/Dec coordinates back to x,y positions and plot matched stars on image

Compare to full catalog and see where differences are

The full source_catalog output is shown with red dots and the matched sources are shown with smaller white dots over the red dots. This shows the sources that were found in the catalog (source_catalog step) but not listed as a match between the input and calculated catalogs.

Look at positions of sources in the simulated images from the catalogs used to create the images

Are there more stars/galaxies in the simulation input catalogs than actually fit on the combined image? That would explain the sources beyond image edges.

Compare fluxes between input sources and output catalog

Input catalog 'combined' has Ks and J columns for stars and flux for galaxies. Output source catalog 'catalog' has aper_total_flux, 'isophotal_flux', 'aper_total_abmag', 'aper_total_vegamag', 'isophotal_abmag' and isophotal_vegamag.

The first plot compares the magnitudes for the input simulated catalog (in blue) and the sources found with source catalog in green.

Compare the star magnitudes and magnitude differences to the x position of the stars.

Plot AB mag vs x position

Simulated stars in blue, sources found in green.

Plot difference in magnitude against x and y position.

Found catalog minus matched simulated AB mag vs. x position and vs. y position, with the median magnitude difference plotted as a horizontal line in red.

Plot AB magnitude difference against AB mag to see if there is a pattern in the differences by mag.

Median difference marked by line in red.

Passing criteria

Look at the matched image to see what sources were matched between the simulated and found catalogs and see if there is a pattern to which 'found' sources were not matched to the simulated catalog. Are they mostly galaxies or were the found sources in areas of confusion between two stars or sources where the source finding algorithm might not have pinpointed the actual centroid of the source. If the majority of the sources matched seem to be the stars that are clearly marked and properly found, that part of the test passes.

If the plots showing magnitudes of the matched sources show that they're close (exact criteria for 'close' TBD), then that test passes.

This test is mostly visual inspection of the images and plots. As long as nothing is obviously wrong (no columns in the table marked as all NaNs, sources found seem to be real sources, and a good number of them match the simulated sources input into the image), then this test passes.

About this Notebook

Authors: M. Cracraft, M. Libralato and T. Temim, MIRI Branch

Updated On: 08/18/2021